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Art Unit: 2655 

DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments filed 6/1 8/2004 have been fully considered but they are 
not persuasive. 

In response to applicant's argument that Examiner's conclusion of obviousness is 
based upon improper hindsight reasoning, it must be recognized that any judgment on 
obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. 
But so long as it takes into account only knowledge which was within the level of 
ordinary skill at the time the claimed invention was made, and does not include , 
knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. 
See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). 

The "combination mode" of Oberteuffer et al. teach combining verbal input Ml 
and handwritten input M2 to produce a third result (FIG. 2, Col. 62-66). Let us analyze 
each of the inputs and the combined output of the Oberteuffer's system. 

I. M1 is produces a set of {several) text strings, letters, characters, or words 
(Col. 5, lines 4-5). In other words, the output of Ml is textual. 

II. M2 is a translation of "hand-printed or cursive writing written by an 
operator using electronic pen into machine encoded text data forming part of the 
electronic document ." (Col. 4, lines 40-44). Therefore, the output of M2 is also textual, 
which suggests that the outputs of Ml and M2 consist of the same types of elements 
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(characters, strings, words, etc). Hence, we have two textual sets that are to be 
combined in step III (as opposed to words from M1 and checkmarks, circles, etc. from 
M2) 

III. Here Oberteuffer et al. teach combining verbal (Ml) and handwritten (M2) 
inputs and producing "an improved recognition of the input speech from additional data 
in mode M2 to select from among recognition candidates identified by mode M1 " (Col. 
5, lines 8-11). In other words, the combination of M1 and M2 (let's call it MC) is limited 
to at most the elements contained in M1 - elements of M2 which are not in Ml cannot 
be in MC since the results are chosen from among the elements of Ml . Therefore, M2 
is either a full subset of Ml or the elements in M2 which are not in Ml are ignored. For 
practical purposes, M2 is within Ml . 

Now let us look at possible operations between the Ml and M2 sets. These are: 
a union of M1 and M2, an intersection of Ml and M2, and a difference of Ml and M2. 
Because M2 is within M1, a union of these sets does not make sense from a practival 
perspective. Therefore, only an intersection of Ml and M2 or a difference between M1 
and M2 can be used for further narrowing down the elements of Ml. Both of these 
operations are equally suggested by the Oberteuffer et al.'s reference and would have 
been obvious to one of ordinary skill in the art. 

In light of the above, it is clear that any practitioner with ordinary skills in the art 
would not need to use applicant's disclosure in order to deduce that the resulting set 
MC is either an intersection or a difference of Ml and M2. Hence no improper hindsight 
reasoning has been used by Examiner. 
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2. Applicant's arguments against the "obviousness" of combinations of Oberteuffer 
et al. and Waibel or Lindliorn et al. are moot, as Claim 1 was not rejected on the basis 
of the above combinations. Claim 1 was rejected as being obvious over Oberteuffer et 
al. in view of Young et al. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claim 1 is rejected under 35 U.S.C. 103(a) as being obvious over Oberteuffer et 
al. (6,438,523), in view of Yong et al. (6,064,959). 

Oberteuffer et al. discloses: 

• utilizing a speech recognition algorithm to generate a first list according to verbal input 
("several text strings, letters, characters') - (elem. 204, FIG. 2, Col. 5, lines 3-6) 

• utilizing a character recognition algorithm to generate a second list according to handwritten 
input - (Col. 5, lines 6-8) 

• generating a third list that is an intersection of characters common to the first list and the 
second list - (Col. 5, lines 8-1 1 ) 
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Also, Oberteuffer et al. disclose an ennbodiment for using both verbal and 
handwritten inputs simultaneously. (FIG. 3 and Col. 5, lines 12-23) 

Oberteuffer et al. do not disclose "presenting the third list" to a user. 

Young et al. teaches a disclosing a list of words to the user and permitting the 
user to choose the correct word from the list (Col, 1 , lines 50-58) 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Oberteuffer et al. as taught by Young et al. in order to 
present the user with a list of possible choices for the recognized word, because it 
would improve the accuracy of the speech recognition process. Here, the user chooses 
the final candidate for the recognized word from a list of possible words and therefore 
avoids the problem of computer incorrectly choosing the final version of the word based 
on incomplete information. 

5. Claims 2-8 are rejected under 35 U.S.C. 103(a) as being obvious over 
Oberteuffer et al. and Young et al., and further in view of Larkey (5,127,055) and 
Carman, II (5,454,046). 

As per claim 2, Oberteuffer et al. disclose a system comprised of a speech 
recognition engine and a cursive handwriting recognition engine (elems. 108, 110, FIG. 
1). 
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Oberteuffer et al. and Young at al. do not disclose a "database from wliich 
cliaracters are selected by the speech recognition algorithm and the character 
recognition algorithm to fill the first list and the second list, respectively". 

Larkey teaches a speech recognition system that "processes and analyzes the 
incoming speech and compares the incoming speech to reference patterns stored in a 
reference pattern storage memory." (Column 4, lines 13-16) 

Carman, II teaches a handwriting recognition system that has "a user specific 
recognition database for storing data pairs" (48, See FIG. 2 and Column 2, lines 41-43) 

At the time of the invention it would have been obvious to a person of ordinary 
skill in the art to modify the recognition engines of Oberteuffer et al. and Young et al. to 
use the databases for word storage, as taught by Larkey and Carman, II. The 
motivation for doing so would have been the improved vocabulary capacity of the 
speech and handwriting recognition systems. 

As per claims 3 and 8, Oberteuffer et al. disclose a system comprised of speech 
recognition engine and cursive handwriting recognition engine (elems. 108, 110, FIG. 1) 

Oberteuffer et al. and Young et al. do not disclose a "adding a first character to 
the database, the first character generated by the user using an auxiliary input method". 

Larkey teaches a speech recognition system that "that features dynamically 
adding new reference patterns to the stored reference patterns during this speech 
recognition process in response to the recognition correction actions and providing such 
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additional reference patterns for use in recognizing new unknown speech input 

utterances." (Column 2, lines 25-30) 

Carman, II teaches a system that "queries the user for textual data and then 
stores a new data pair "thus improving subsequent recognition by virtue of an 
augmented user specific sample recognition database file" (Column 2, line 62 - 
Column 63, line 6). Carman, II also teaches the use of keyboard (28, FIG. 1). 

At the time of the invention it would have been obvious to a person of ordinary 
skill in the art to modify the recognition engines of Oberteuffer et al, and Young et al. to 
use databases that can store additional user input, as taught by Larkey and Carman. II. 
The motivation for doing so would have been an ability to "train" the recognition system 
in Oberteuffer et al. to understand new words or characters. Additionally, at the time of 
the invention it would have been obvious to a person of ordinary skill in the art that if 
initially the vocabulary set stored in the database was empty, the users would have to 
"train" the recognition system by adding new words/characters to the empty database 
through keyboard, as taught by Carman, until the database contained sufficiently large 
number of words/characters for the proper operation of the recognition system. 

As for claims 4-5, Oberteuffer et al. disclose a system comprised of speech 
recognition engine. (108, FIG. 1) 

Oberteuffer et al. and Young et al. do not disclose a system where "speech 
recognition algorithm utilizes a first standard for speech recognition, and adapts the first 
standard to verbal characteristics of the user" 
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Oberteuffer et al. also do not disclose a system where "characteristics of the user 
corresponding to the first character are added to the database" 

Larkey teaches a speech recognition system that "that features dynamically 
adding new reference patterns to the stored reference patterns during this speech 
recognition process in response to the recognition correction actions and providing such 
additional reference patterns for use in recognizing new unknown speech input 
utterances." (Column 2, lines 25-30) 

At the time of the invention it would have been obvious to a person of ordinary 
skill in the art to modify the speech recognition engine of Oberteuffer et al. and Young et 
al. to use new input from the user as a standard and to store it in the database for future 
reference, as taught by Larkey. The motivation for doing so would have been an ability 
to "train" the speech recognition system to "learn" new characters and words, thus 
adjusting to the idiosyncrasies of each user. 

As for claims 6-7, Oberteuffer et al. disclose a system comprised of handwriting 
recognition engine. (110, FIG. 1) 

Oberteuffer et al. and Young et al. do not disclose a system where "the character 
recognition algorithm utilizes a second standard for character recognition, and adapts 
the second standard to handwriting characteristics of the user." 
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Oberteuffer et al. and Young et al. also do not disclose a system where "the 
handwriting characteristics of the user corresponding to the first character are added to 
the database." 

Carman, II teaches a system that "queries the user for textual data and then 
stores a new data pair ," "thus improving subsequent recognition by virtue of an 
augmented user specific sample recognition database file" (Column 2, line 62 - 
Column 63, line 6). 

At the time of the invention it would have been obvious to a person of ordinary 
skill in the art to modify the handwriting recognition engine of Oberteuffer et al. and 
Young et al. to use new input from the user as a standard and to store it in the 
database for future reference, as recited by Carman II. The motivation for doing so 
would have been an ability to "train" the handwriting recognition system to "learn" new 
characters and words, thus adjusting to the idiosyncrasies of each user. 

Conclusion 

6. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dmitry Brant whose telephone number is (703) 305- 
8954. The examiner can normally be reached on Mon. - Fri. (8:30am - 5pm). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Talivaldis Ivars Smits can be reached on (703) 306-301 1 . The fax phone 
number for the organization where this application or proceeding is assigned is (703) 
872-9306. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to Tech Center 2600 receptionist whose telephone 
number is (703) 305- 4700. 



DB 

9/2/04 




DANIEL ABEBE 
PRIMAWEXAMWER 



